Unsupervised learning of an embodied representation for action selection

We propose a principle on how a computational agent can learn the structure of a classic discrete state space. The idea is to do a kind of principal component analysis on a matrix describing transitions from one state to another. This transforms the space of discrete, completely separate, states into a dimensional representation in a Euclidean space. The representation supports action selection, ideally turning action selection into a trivial problem: the route to a goal state can be directly obtained from the representation. Thus, the computations typically performed by dynamic programming and reinforcement learning are largely replaced by learning the representation. This has the benefit that the representation is not dependent on which state happens to be the goal state; thus, change of goal does not necessitate re-learning, which is in stark contrast to classic reinforcement learning theory.